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1 . BACKGROUND 


The  Coast  Guard  has  developed  a Task  Bank  of  tasks  performed  by  bridge, 


cargo,  and  engineering  personnel  of  commercial  vessels  with  a view  towards 


using  the  data  contained  therein  to  develop  licensing  programs 


that  insure  the  highest  degree  of  safety  in  ship  operation.  Functional 


Job  Analysis  (FJA),  a method  developed  by  Sidney  A.  Fine,  was  used  to  pro- 


duce the  Task  Bank  of  some  800  tasks. 


FJA  is  both  a conceptual  system  and  a method  of  analysis.  Central  to 


the  method  of  analysis  is  the  role  of  language,  since  all  data  (whether 


obtained  from  observation,  interview,  or  written  material)  is  converted  to 


language.  Hence  reliability  is  completely  dependent  on  how  the  analytic 


information  is  expressed.  Thus,  FJA  methodology  is  centered  on  controlling 


the  language  used  to  describe  work.  It  does  so  by  using  the  structure  of 


the  sentence  as  the  vehicle  for  communicating  the  information  (see  Fig.  1), 


and  by  defining  a vocabulary  of  functional  activity  (see  Fig.  2).  A task 


statement  is  formulated  in  response  to  five  questions: 


• Who?  (Subject)  The  subject  of  a task  statement  is  understood 
to  be  simply  worker.  The  task  statement  does  not  define  what 
kind  of  worker. 


Performs  what  action?  (Action  verb)  A task  statement  requires 
a concrete,  explicit  action  verb.  Verbs  which  point  to  a pro- 
cess (such  as  develops,  prepares,  interviews,  counsels,  evaluates, 
and  assesses)  should  be  avoided  or  used  only  to  designate  broad 
processes,  methods,  or  techniques  which  are  then  broken  down 
into  explicit  action  verbs. 


^The  Task  Bank  was  developed  under  contract  DOT  CG-4-903A  by  professional  re- 
searchers at  Operations  Research,  Inc.,  Silver  Spring,  Md.  These  workers 
had  received  one  week's  training  in  Functional  Job  Analysis  from  Dr.  Fine 
after  which  they  began  an  actual  task  analysis  drawing  on  experience,  ex- 
isting literature,  and  observations  in  the  field. 
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• To  accomplish  what  immediate  result?  The  purpose  of  the 
action  performed  must  be  explicit  so  that  (1)  its  relation 
to  the  objective  is  clear  and  (2)  performance  standards  for 
the  worker  can  be  set. 

t With  what  tools,  equipment,  or  work  aids?  A task  statement 
should  identify  the  tangible  instruments  a worker  uses  as 
he  performs  a task;  for  example,  telephone,  typewriter,  pen- 
cil/paper, checklist,  written  guides,  etc. 

• Upon  what  instructions?  A task  statement  should  reflect  the 
nature  and  source  of  the  instruction  the  worker  receives. 

It  should  indicate  what  in  the  task  is  prescribed  by  a 2 

superior  and  what  is  left  to  the  worker's  discretion  or  choice.^ 

Tasks  formulated  in  this  manner  and  responsive  to  these  questions  provide 

the  information  necessary  It  consistently  and  reliably  determine  the  com- 

plexit  1 of  difficulty)  and  orientation  (worker  involvement  as  a 

pers  1C  task  and  to  develop  performance  standards  and  training 

content.  Each  of  the  elements  of  the  task  analysis  reinforces  the  others. 

The  interrelationships  among  the  parts  is  illustrated  by  this  paradigm: 

In  Order  To  Perform  This  Task  - as  described  by  the  task  statement  and 

supported  by  the  ratings  - To  These  Standards,  the  Worker  Needs  This 

Training.  (See  Fig.  3). 

The  reliability  of  FJA  task  statements  is  determined  by  establishing 
agreement  among  analysts  on  the  ratings  made  from  seven  Guttman  type 
(ordinal)  scales  containing  six  to  eight  levels.  The  seven  scales  measure 
functional  performance  as  such  performance  relates  to  Things,  Data,  People, 
Prescription/Discretion,  Reasoning,  Math  and  Language.  In  addition,  an 


^Fine,  Sidney  A.  and  Wiley,  Wretha,  Introduction  to  Functional  Job  Analysis, 

Kalamazoo,  Michigan:  W.  E.  Upjohn  Institute  for  Employment  Research,  1971, 

pp.  10-11. 

^Fine,  Sidney  A.,  Functional  Job  Analysis  Scales,  Kalamazoo,  Michigan: 

W.  E.  Upjohn  Institute  for  Employment  Research,  1973. 
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estimate  of  the  Worker  Function  Orientation  which  measures  the  relative 
importance  in  the  task  of  the  performance  standards  relating  to  Things, 

Data,  and  People,  is  also  made.  (Hov.rvtr,  the  Orientation  assignment  is 
not  an  i.iiportant  consideration  in  the  usual  reliability  study.)  Agreement 
is  judged  to  be  satisfactory  when  a minimum  of  75%  of  the  ratings  (there 
are  seven  ratings  per  task)  are  the  same  or  no  more  than  one  scale  level 
different  between  independent  ratings  of  individual  FJA  analysts.  Signifi- 
cant differences  between  analysts  or  groups  of  analysts  are  evidence  of  an 
unreliable  task  statement  indicating  the  need  for  further  information, 
greater  specificity,  and  improved  clarity  in  the  form  of  the  task  statement. 

The  reliability  of  the  task  statements  is  the  most  critical  factor 
determining  their  value  to  the  Coast  Guard  and  to  the  industry.  It  is 
essential  that  a between-groups  reliability  study  be  completed  before  fur- 
ther use  is  made  of  FJA  under  contract  CG-41-903A.  That  is,  ratings 
assigned  by  FJA  analysts  knowledgeable  of  the  specific  content  in  the  tasks 
must  be  compared  to  ratings  determined  by  FJA  analysts  who  are  not  familiar 
with  the  specific  content  of  the  tasks.  This  reliability  data  is  necessary 
to  evaluate  the  usefulness  of  FJA  for  future  contract  work  and  correct  any 
deficiencies  which  may  exist  in  the  past  and  current  application  of  FJA. 

Results  from  such  a study  may  improve  considerably  the  cost  effectiveness 
and  reliability  of  further  research.  In  order  to  provide  this  data,  Sidney 
A.  Fine  Associates,  Inc.  contracted  to  conduct  a Reliability  Study. 

2.  PROCEDURE 

a)  Reliability 

The  tasks  in  the  Task  Bank  had  been  written  and  analyzed  by  FJA  analysts 
who  were  knowledgeable  about  the  specific  content  of  the  tasks.  In  the  relia-  : 

bility  study,  the  tasks  were  rerated  by  FJA  analysts  from  Fine  Associates  who  ,| 
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1 

■ has  not  been  involved  in  the  development  of  the  Task  Bank  and  who  were  not 

familiar  with  the  specific  nature  of  the  work  analyzed.  The  analysts  studied 
some  background  material  to  acquaint  them  with  the  technical  terminology  em- 
ployed by  the  Coast  Guard.  They  were  then  provided  with  a complete  set  of 
the  task  bank  of  822  tasks  with  all  the  ratings  blanked  out.  Each  of  three 
raters  was  provided  with  a portion  of  the  blanked  task  bank.  At  the  end  of 
pre-arranged  periods  of  time  (a  week  or  ten  days)  the  material  was  exchanged 
for  another  portion  on  which  that  rater  had  not  yet  worked.  Therefore,  two 
analysts  independently  rated  each  task  in  the  Task  Bank.  These  two  ratings 
were  compared  and  where  disagreements  occurred,  a third  analysts  also  rated 
the  task.  Differences  in  the  ratings  were  resolved  through  a group  dis- 
cussion and  by  arriving  at  a consensus.  Only  then  were  the  Associates' 
agreed  on  ratings  for  the  seven  scales  compared  and  evaluated  against  the 
original  ratings  to  calculate  the  reliability  and  to  identify  and  determine 
the  nature  of  any  differences  between  them. 

The  seven  ratings  referred  to  are  those  which  appear  in  Figure  3 on  page 
5 of  this  report.  These  seven  ratings  deal  with  the  involvement  of  the  task 
with  Data,  People  and  Things,  the  scale  of  Worker  Instructions  (the  mix  of 
prescription/discretion)  and  the  General  Educational  Development  represented 
by  the  Reasoning,  Math  and  Language  scales.  They  do  not  include  the  orienta- 
tion ratings.  The  seven  scales  accurately  measure  the  levels  necessary  to 
carry  out  the  paradigm  represented  in  Figure  3:  To  do  this  task  to  these 

standards,  the  worker  needs  this  training.  In  assessing  the  reliability  of 
the  tasks,  the  same  seven  scale  ratings  assigned  to  a task  should  be  assigned 
by  an  independent  group  of  raters.  A difference  of  only  one  level  in  any  of 
the  scales  is  not  significant  and  therefore  constitutes  agreement. 


The  differences  between  the  Associates  ratings  and  the  original  raters 
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were  of  three  sorts.  The  first  related  to  the  central  focus  of  this  study, 
namely  differences  in  ratings.  The  second  involved  noting,  in  the  process  of 
reviewing  the  ratings,  some  departures  from  the  principles  and  techniques  of 
FJA  Methodology.  Finally,  sometimes  the  information  in  the  task  "sentence", 
performance  standards,  and  the  training  content  was  ambiguous  or  inconsistent, 
blurring  the  meaning/intent  of  the  task.  The  latter  two  differences  could 
occur  even  when  ratings  were  the  same  as  between  the  Associates  and  the 
original  raters. 

These  three  kinds  of  differences  were  dealt  with  as  follows  in  making 
changes.  Sometimes  the  Associates  new  ratings  of  the  seven  scales  not  only 
resulted  in  changes  of  two  or  more  levels  but  also  had  the  effect  of  creating 
differences  in  the  orientation  pattern,  that  is,  the  relative  involvement  with 
Things,  Data,  and  People*.  (These  orientation  changes  were  not  part  of  the 
reliability  count.)  Such  changes  were  discussed  by  the  Associates  and  if  the 
ambiguities  were  cleared  up  among  them,  the  original  ratings  were  accepted. 

In  the  other  two  instances  apparent  inconsistencies  or  ambiguities  in  the 
information  or  inadequate  support  for  the  ratings  were  noted  and  changes 
made. 

There  are  some  basic  guidelines  used  in  performing  an  FJA  study.  One 
key  item  to  look  for  is  a range  of  no  more  than  one  level  difference  between 
Data,  Worker  Instructions  and  Reasoning.^  These  three  items  are  very  closely 
related  and  a rating  of  4 on  Data,  6 on  Worker  Instructions  and  5 on  Reasoning, 
for  example,  indicates  that  the  original  rater  may  not  fully  understand  the 
principles  of  FJA. 

* Orientation  categories  are  High,  Medium  and  Low.  High  includes  a range  of  90% 
to  65%;  Medium  includes  a range  of  60%  to  35%;  Low  includes  a range  of  30%  to 
5%. 

1.  Fine,  Sidney  A.,  Ann  M.  Holt  and  Maret  Hutchinson.  Functional  Job  Analysis: 
How  to  Standardize  Task  Statements,  the  W.  E.  Upjohn  Institute  for  Employment 
Research,  October  1 974 , pp . 17,  26 . 
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Another  item  to  watch  is  a rating  of  2 on  the  People  scale.  A 2 rating 
represents  an  exchange  of  information.  If  the  task  statement,  performance 
standards  and  training  content  make  no  mention  of  any  exchange  of  information 
between  the  worker  and  another  person,  a rating  of  lA  or  IB  is  correct. 

Both  of  the  above  are  worthy  of  mention  here  even  though  the  difference 
between  the  original  ratings  and  the  new  ratings  may  be  only  1 level;  the  point 
is  that  some  1 level  differences  are  a little  more  important  than  others.  These 
are  some  of  the  items  addressed  in  the  report.  However,  a reliability  of  98.1% 
was  achieved  based  on  only  the  2-level  differences.  That  is,  differences  in 
ratings  of  only  one  scale  level  or  within  the  same  orientation  category  which 
were  consistent  with  the  methodology  of  the  task  were  accepted.  (See  Table  1, 
page  12. ) 

In  examining  the  scale  level  differences  we  (the  independent  raters)  must 
ask  ourselves  what  it  is  about  the  task  statement  that  caused  us  to  rate  it 
differently.  This  is  where  our  group  discussion  comes  in  as  outlined  on  page  7. 
Sometimes  the  group  discussion  elicits  the  realization  that  a word  or  phrase 
in  the  task  statement  was  ambiguous  or  too  general  in  nature  which  led  to  some 
disagreement  on  our  part.  If  we  are  able  to  identify  and  clarify  this  ambiguity 
and  agree  that  the  original  rating  was  reasonable,  no  changes  were  made.  In 
some  cases  we  made  suggestions  as  to  how  to  clarify  the  wording  to  more  ac- 
curately match  the  rating.  In  other  cases  we  may  have  felt  that  the  rating 
was  not  accurate  and  needed  to  be  changed. 


The  criterion  of  75%  reliability  is  basically  founded  on  an  examination 


of  the  seven  scales;  the  general  orientation  assigned  to  the  task  by  the  raters 
is  not  rigidly  scrutinized  in  a reliability  study. 
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In  initially  assigning  a percentage  to  reflect  the  overall  orientation 
of  the  task,  the  general  guidelines  are  reflected  in  the  footnote  on  page  8 of 
the  report.  In  checking  the  completed  task  bank,  a slight  shift  in  percentages 
which  does  not  alter  the  basic  assignment  within  the  range  of  High,  Medium  or 
Low  would  be  considered  insignificant. 

To  summarize,  according  to  the  basic  guidelines  for  establishing  reliability, 
98.1%  of  the  ratings  met  the  criterion  of  difference  of  no  more  than  1 level. 

This  98.1%  should  be  compared  to  the  established  rule  that  75%  agreement  means 
reliability  has  been  achieved. 

2)  Additional  Comments 

After  rating  the  tasks  which  were  in  disagreement  (two  levels  different 
on  any  of  the  seven  scales),  the  Associates  also  re-examined  those  tasks 
which  contained  one  or  more  one-level  differences.  After  some  discussion 
during  which  the  criticality  of  safety  factors  in  the  performance  standards 
and  training  content  was  considered  at  length,  it  was  decided  to  provide  the 
Coast  Guard  with  a much  more  detailed  analysis  applying  harsher  criteria. 

Some  of  the  tasks  with  one  level  differences  were  then  added  to  the  original 
group  of  task  statements  which  were  "not  in  agreement".  However,  these  are 
not  included  in  the  calculation  of  reliability. 

All  parts  of  the  tasks,  including  the  orientation,  were  examined  in 
detail.  By  the  time  the  task  had  been  sifted  through  a second  time,  the 
Associates  found  they  had  pulled  out  a total  of  188  of  the  822  tasks  which 
were  found  to  have  some  type  of  problem  which  suggested  changes  or  comments. 

These  additional  data  are  included  in  Tables  2 and  3 in  Section  3,  Results. 

Appendices  A-G*  include  all  those  tasks  which  required  clarification, 
rewriting,  or  other  changes.  To  facilitate  the  comparison  of  the  differences 


* See  Table  of  Contents,  p.  ii,  for  full  titles  of  the  appendices  which  were 
delivered  in  March  1977  to  the  Coast  Guard. 
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between  the  original  ratings  and  the  Associate's  ratings,  only  those  tasks 
which  have  changes  are  included.  Both  sets  of  ratings  appear  on  each  task. 
Explanations  of  the  differences  as  well  as  suggestions  for  clarification 
are  included  on  a comment  sheet  which  directly  follows  each  task. 

3.  RESULTS 

1)  Reliability 

As  previously  noted,  satisfactory  agreement-reliability-is  achieved  when 
at  least  75%  of  the  ratings  of  two  independent  groups  of  analysts  are  the 
same  or  no  more  than  one  scale  level  different.  Since  there  are  822  tasks, 
the  total  number  of  ratings  is  7 x 822  or  5754.  Six  hundred  thirty  four 
(634)  tasks  involving  4438  ratings  were  rated  the  same  or  only  1 level 
different;  no  changes  were  made  on  those  tasks.  One  hundred  seven  (107) 
ratings  were  found  to  be  in  disagreement,  out  of  a possible  5754.  This 
yields  a reliability  of  98.1%. 

2)  Additional  Comments 

An  additional  61  tasks  were  then  pulled  in  accordance  with  the 
Associates'desire  to  make  further  constructive  criticism.  Of  these  tasks 
another  133  one-level  rating  changes  were  indicated  (see  Table  2).  Suggestions 
for  rewording  also  have  been  indicated  to  more  clearly  support  the  ratings. 

As  noted,  the  Associates  indicated  suggested  changes  on  a total  of  188  task 
statements.  Of  these  188  tasks  where  changes  were  made  (involving  1316 
ratings)  1076  ratings  were  not  changed.  Thus,  4438  plus  1076  for  a total  of 
5514  ratings  remained  unchanged.  This  is  95.8%  of  the  total  and  represents 
the  adjusted  reliability  based  on  scale  rating  changes  including  some  1-level 
changes. 
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The  188  tasks  that  the  Associates  suggest  be  changed  involved  240  scale 
rating  changes.  These  changes  included  some  1 level  differences  in  ratings 
as  noted  in  Section  2 above.  These  changes  have  been  suggested  when  the 
original  ratings  were  found  to  be  inconsistent  with  the  FJA  methodology  or 
the  intent  of  the  task.  In  addition,  for  about  36  of  these  188  tasks  there 
were  major  shifts  in  the  orientation.  On  19  tasks  the  original  ratings  were 
left  unchanged  but  some  verbal  changes  were  made  to  suggest  how  the  original 
ratings  might  be  better  supported.  However,  in  accordance  with  standard 
procedures  only  those  changes  indicated  in  Table  1 need  to  be  made.  Tables 
2 and  3 are  additional  reconriendations  which  the  Associates  have  made.  We 
particularly  recommend  the  changes  indicated  as  "Major  orientation  shifts/ 
rewrite"  appearing  in  Table  3. 
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TABLE  1 

SUMMARY  OF  TWO-LEVEL  RATING  CHANGES  BY  APPENDIX  LETTER 
IN  ACCORDANCE  WITH  ESTABLISHED  RELIABILITY  METHODS 


I 

. £ 

[I 


k 

I 

i 

I 

i 


Appendix  Letter 


A 

B 

C 

D 

E 

F 

G 

Totals 

1.  Two  or  more  level 
ratings  changed 

1 

30 

25 

30 

12 

5 

4 

107 

Number  of  possible 
changes  (7  per  task) 

245 

1211 

1022 

1169 

686 

861 

560 

5754 

Number  of  tasks  in 
disagreement 

1 

16 

15 

20 

6 

5 

4 

27 

Number  of  tasks  in 
category 

35 

173 

146 

167 

98 

123 

80 

822 

Reliability  (%) 

99.5 

97.5 

97.5 

97.4 

98.2 

99.4 

99.2 

98.1 

TABLE 

2 

SUMMARY  OF  ONE-LEVEL  SUGGESTED 

CHANGES  BY  APPENDIX 

LETTER 

2.  One-level  ratings 
changed 

4 

29 

28 

25 

1 

22 

24 

133 

Total  rating  changes 
(Including  Table  1) 

5 

59 

53 

55 

13 

27 

28 

240 

♦Adjusted  reliability  (5^) 

97.9 

95.1 

94.8 

95.2 

98.1 

96.8 

95.0 

95.8 

TABLE  3 

SUMMARY  OF 

OTHER  CHANGES 

BY  APPENDIX 

LETTER 

Appendix  Letter 

Number  of  major 
orientation 

A 

B 

C 

D 

E 

F G 

Totals 

shifts/rewrite 

T 

4 

9 

18 

1 

■g  nr 

5T 

Number  of  minor 

orientation 

shifts/rewording 

8 

22 

17 

13 

7 

6 

73 

Total  other  changes 

9 

26 

26 

31 

8 1 

14  10 

124 

♦These  percentages  reflect  a hypothetical  reliability  based  on  two-level  changes 
plus  those  one-level  changes  the  Associates  suggest  be  made  in  order  to  achieve 
PERFECT  agreement  on  ratings. 
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While  the  level  of  agreefnent  indicates  the  reliability  of  the  task  data, 
the  study  identified  a number  of  trends  which  should  be  considered  in  evaluating 
the  applicability  of  the  Task  Bank  to  meet  specific  needs.  The  conment  sheet 
following  each  task  statement  in  the  appendices  goes  into  detail. 

3)  Task  Statement 

Each  task  statement  must  include  sufficient  information  to  clearly  portray 
the  task  as  it  is  performed.  Task  statements  should  be  written  using  clear 
action  verbs  to  describe  what  the  worker  does.  In  reviewing  the  Task  Bank,  the 
Associates  found  that  the  worker  actions  were  often  described  only  by  broad  | 

process  verbs  such  as  "tests"  or  "monitors"  (see  C0-III.D.3  and  C0-II.C.5  \ 

i 

i 

following).  In  addition,  some  task  statements  tended  to  ignore  two  questions  j 

from  the  model  sentence  worksheet  for  task  statements:  "using  what  tools, 

equipment,  and  work  aids?"  and  "upon  what  instructions?"  (See  Figure  1.) 

Answers  to  these  questions  would  clarify  the  ambiguous  verbs  such  as  "tests" 
or  "monitors"  in  terms  of  the  level  of  complexity  and  the  prescription/discretion 
mix  required.  For  this  study,  it  was  possible  to  obtain  this  information  from 
other  elements  of  the  task  analysis  - the  performance  standards  and  the  train- 
ing content.  It  should  be  stressed,  however,  that  in  order  to  communicate 
precisely  about  the  work  it  is  necessary  to  carefully  read  the  entire  task, 
considering  the  ratings,  orientation  assignments,  performance  standards  and 
training  content. 

4)  Worker  Function  Levels  and  Orientation 

Most  of  the  original  ratings  on  the  worker  function  scale  level  were  in 
agreement  with  the  Associates'  ratings.  However,  it  was  noted  that  when  the 
task  statement  referred  specifically  to  emergency  or  safety  procedures,  the 
Data,  Worker  Instruction,  and  Reasoning  Scale  ratings  tended  to  be  higher 
than  for  a similar  or  comparable  task.  (See  III.B.5  and  III. B. 10.)  People 
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1 


merit  did  not  describe  any  "exchange  of  information"  (see  C0.1.A.2  and  Nuke 
2.1.1.).  The  5%  orientation  correctly  reflects  the  fact  that  there  is  no 
interpersonal  involvement  in  this  task. 

The  orientation  assignments  reflect  the  relative  involvement  expressed 
in  proportions  of  100%  of  the  work  with  Data  (information  and  ideas),  People 
(interpersonal  communication)  and  Things  (physically  handling  tangibles). 
There  was  a tendency  for  the  original  orientation  assignment  to  reflect  less 
data  involvement  than  described  or  indicated  by  the  task  statement,  scale 
ratings,  performance  standards  and/or  training  content  (see  III-E.8).  The 
orientation  assignments  also  tended  to  reflect  a confusion  between  things 
which  are  physcially  handled,  manipulated  or  operated  and  things  which  are 
a source  of  Data,  e.g.  data  readouts  or  signals  on  computers  or  other 
electronic  equipment. 

Discrepancies  of  1 level  occurred  frequently  in  the  Language  scale. 

While  this  is  within  the  bounds  of  defined  reliability,  there  appeared  to 
be  inconsistencies  in  the  ratings  by  the  original  raters.  The  basis  for 
the  ratings  appeared  to  be:  a)  procedural  manuals  such  as  the  Operations 

and  Safety  Manual  (see  CO-III.A.l)  and  b)  "prescribed"  or  "specified 
procedure".  The  original  ratings  were  usually  a level  4 or  a level  3 but 
not  consistently  one  or  the  other  (see  C0-I.B.14).  On  a fews  tasks  a 
level  2 was  used  for  specified  procedures  (see  C0-I.B.5).  On  some  tasks 
a level  4 was  assigned  without  firm  support  in  the  task  statement  (see 
CO-1. B. 15).  The  Associates'  ratings  tended  to  be  level  4 where  specified 
procedures  are  cited.  The  Associates'  consistencies  and  the  original  raters' 
inconsistencies  produced  many  1 level  discrepancies.  This  was  established 
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during  review  of  the  first  200  tasks  and  verified  during  subsequent  rating. 

The  inconsistencies  can  easily  by  cleared  up  by  careful  analysis  of  the  basic 
reference  materials. 

The  Math  level  produced  some  similar  problems  where  only  a careful  study 
of  the  entire  task  produced  any  support  for  the  original  ratings  (see  CO-III.H.l). 

5)  Training  Content 

The  training  content  portion  of  the  task  should  contain  only  those  items 
required  to  perform  the  particular  task.  The  Associates  felt  that  some  tasks 
contained  training  content  for  the  job  as  a whole,  rather  than  for  a specific 
task  (see  C0-II.C.4  and  1.4.5).  This  often  occurs  when  task  statements  are 
written  to  cover  the  tasks  entailed  in  a specific  job.  One  possible  result 
of  this  is  selection  of  personnel  who  are  over-qualified  for  the  tasks  at  hand. 

4.  SUMMARY 

The  reliability  of  the  ratings  on  the  822  tasks  was  very  high.  The 
Associates'  impression  was  that  most  of  the  necessary  data  was  available  in 
the  task  analysis;  the  format,  however,  was  not  always  consistent  with  the 
principles  of  FJA.  If  feasible,  one  of  the  original  FJA  analysts  thoroughly 
familiar  with  the  content  could  review  the  entire  task  bank  for  consistency, 
as  it  was  obvious  that  different  analysts  were  at  work  on  specific  sections. 

The  Language  inconsistency  is  a salient  example.  Many  of  the  task  statements 
can  be  polished  by  drawing  on  the  performance  standards  and  training  content 
data  to  include  a phrase  or  two  for  clarification.  Suggestions  along  this 
line  have  been  indicated  on  some  of  the  comment  sheets  appended  to  the  tasks 
in  the  March  report.  Because  of  the  frequency  of  this  occurrence  and  lack  of 
knowledge  of  specific  content  the  Associates  did  not  feel  it  was  within  the 
scape  of  their  contract  to  make  these  changes. 
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WORKER  FUNCTION  LEVEL  AND  ORIENTATION  GENERAL  EDUCATIONAL  DEVELOPMENT 


-All  significant  changes  In  plant  capabilities  are  -Knowledge  of  ship's  watchstanding  procedures, 

reported  to  appropriate  personnel  without  exception.  -Knowledge  of  ship’s  nuclear  power  plant  systems. 

-Knowledge  of  ship's  nuclear  power  plant  equipment  or 
system  malfunctions  effect  on  overall  propulsion  capabili- 
ties of  the  ship. 


SIDNEY  A.  FINE  ASSOCIATES,  INC. 


By  going  a step  further,  the  Associates  have  identified  some  specific 
areas  for  improvement.  However,  again,  the  figure  which  should  be  compared 
are  98.1%  reliability  achieved  versus  a goal  of  75%. 

A detailed  critique  by  the  Associates  is  not  usually  given  if  there  is 
reliability.  In  striving  to  meet  the  spirit  and  not  just  the  letter  of  the 
study,  the  Associates  have  attempted  to  offer  their  suggestions  for  improve- 
ment. The  overall  work  performed  by  the  original  raters  was  good.  Our 
suggestions  are  aimed  at  moving  closer  to  perfection  since  we  recognize  that 
safety  is  the  most  important  factor  in  the  tasks.  This  is  the  factor  that 
caused  us  to  go  far  beyond  the  normal  work  entailed  in  performing  the  re- 
liability study. 


5.  TYPES  OF  CHANGES 

The  final  section  of  this  report  consists  entirely  of  a detailed  list 
of  the  changes  by  task  number. 

TYPES  OF  CHANGES 


Task  No.  *Disaqreement  **Agreement 


APPENDIX 

_A  - Engineering  Personnel  of  Nuclear  Ships 

1.1.4 

Orientation 

1.1.6 

Orientation;  rewording 

1.3.1 

One  1-level;  orientaiton 

1.3.3 

Rewording 

1.3.4 

One  1 -level 

1.3.5 

Orientation 

1.4.1 

One  3-level 

2.2.1 

Rewording 

2.2.2 

Rewrite  - (two  tasks) 

3.1.1 

One  1 -level  , 

3.1.3 

One  1 -level  I 

3.2.3 

Orientation 

♦One  or 

1 

more  2-level  changes/major  orientation  shift/rewrite  ' 

♦♦One  or 

more  1-level  changes/minor  orientation  shift/rewording 

j 
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TYPES  OF  CHANGES  (cont'd) 


Task  No. 

♦Disagreement 

♦♦Agreement 

APPENDIX  B 

- Cargo  Operations  of  a 

Liquefied  Natural  Gas  Ship 

CO- 

I.B.12 

Rewording 

I.C.l 

Orientation 

I.C.3 

Orientation 

II. B. 8 

Rewording 

II.B.12 

Rewording 

II.C.l 

Orientation 

II. C. 3 

Orientation 

II.D.l 

One  2-level 

lI.E.l 

Orientation;  one  1-level 

III. A. 2 

Orientation 

III.F.3 

One  2-level 

"Things"  Sub-Category 

III.F.4 

Three  2-level 

III. 1.4 

Rewrite 

IV. A. 1 

Two  2-level 

IV.B.l 

Two  2-level 

IV.C.1 

Two  2-level 

IV.D.1 

Two  2-1eve1 

IV.E.l 

Two  2-level 

IV.F.l 

Two  2-level 

IV.G.l 

Two  2-level 

IV.H.l 

Two  2-level 

IV. 1. 1 

Two  2-level 

IV.J.l 

Two  2- level 

IV.K.l 

Two  2-level 

IV.L.l 

Two  2-level 

V.B.2 

One  1-level 

V.B.4 

Rewording 

V.D.l 

Two  1-level;  orientation 

V.E.1 

Two  1-level;  orientation 

V.F.l 

Two  1-level;  orientation 

VI. A. 1 

Two  1-level;  orientation 

VI. A. 4 

One  2-level 

One  1-level 

VI. A. 5 

Four  1-level;  orientation 

VI.B.1 

Two  1-level;  orientation 

VI. B. 4 

Rewrite 

VI. B. 5 

Four  1-level;  orientation 

VI.C.l 

Three  1-level;  orientation 

VI. C. 3 

Four  1-level;  orientation 

VII. A.2 

One  1-level;  orientation 

VII. A. 6 

Rewrite;  complete  the  ratings 

VII. B. 2 

Orientation 

♦One  or  more  2-level  changes/major 

orientation  shift/rewrite 

♦♦One  or  more  1-level  changes/minor  orientation  shift/rewording 
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TYPES  OF  CHANGES  (cont'd) 

Task  No.  *Disagreement  **Aqreement 

APPENDIX  C - Cargo  Operations  of  an  Unmanned  Liquefied  Natural  Gas  Barge 


TM- 


I. A. 3 

Rewording 

I.B.ll 

Rewording 

I.C.l 

Orientation 

I.C.3 

Orientation 

II. A. 2 

Rewording 

II. B. 7 

Rewrite 

II.B.ll 

Rewording 

II.C.l 

Orientation 

II. C. 3 

Orientation 

II.D.l 

One  2-level 

lI.E.l 

Orientation 

III. A. 3 

One  2-level 

"Things"  sub-category ;ori 

III.D.4 

Rewrite 

IV. A. 1 

Two  2-level 

IV.B.l 

Two  2-level 

IV.C.l 

Two  2-level 

IV.D.l 

Two  2-level 

IV.E.l 

Two  2-level 

IV.F.l 

Two  2-level 

IV.G.l 

Two  2-level 

IV.H.l 

Two  2-level 

IV. 1. 1 

Two  2-level 

IV.J.l 

Two  2-level 

V.B.4 

One  2-level 

Two  1-level 

V.D.l 

Two  1-level;  orientation 

V.E.l 

Two  1-level;  orientation 

V.F.l 

Two  1-level;  orientation 

VI. A. 1 

Two  1-level;  orientation 

VI. A. 4 

One  2-level 

One  1-level;  orientation 

VI. A. 5 

Four  1-level;  orientation 

VI.B.l 

Two  1-level;  orientation 

VI. B. 4 

Rewrite 

VI. B. 5 

Four  1-level;  orientation 

VI.C.l 

One  3-level 

VI. C. 3 

Four  1-level;  orientation 

VII. A. 2 

One  1-level;  orientation 

VII. A. 6 

Rewrite 

VII. B. 2 

Orientation 

♦One  or  more 

2-level  changes/major 

orientation  shift/rewrite 

**0ne  or  more  1-level  changes/minor  orientation  shift/rewording 


-33- 


I 


SIDNEY  A.  FINE  ASSOCIATES,  INC. 


TYPES  OF  CHANGES  (cont'd) 


Task  No.  *Disagreement  **Agreement 

APPENDIX  D - Bridge  Personnel  on  Tankers  and  Freighters 


I. A. 5 

One  2-level 

Two  1-level;  orientation 

I. A. 6 

One  2-level 

"Data"  Sub-category 

I.B.l 

One  2-level  or  reword 

I.B.6 

Orientation 

I.B.7 

Orientation 

I.B.8 

Orientation 

II. A. 1 

Orientation 

II. A. 5 

Two  2-level 

Two  1-level;  orientation 

II. B. 4 

Two  2-level 

II. A. 6 

Orientation 

Four  1-level 

II. A. 8 

One  2-level 

One  1-level 

II. B. 8 

One  3-level;  orientation 

Two  1-level 

II.B.IO 

Orientation 

II.B.ll 

Orientation 

One  1-level 

II. C. 4 

Two  2-level 

II.C.IO 

Orientation 

Two  1-level 

II.C.12 

Orientation 

II.C.13 

Orientation 

One  1-level 

II. D. 4 

Two  2-level 

II.D.13 

Orientation 

II.D.14 

Orientation 

One  1-level 

III. A. 7 

Two  2-level 

III. A. 10 

One  3-level;  orientation 

Two  1-level 

III. A. 12 

Orientation 

III.B.7 

Two  2-level 

III. B. 12 

One  3-level;  orientation 

Two  1-level 

III. B. 14 

Orientation 

III.C.7 

Two  2- level 

III. C. 13 

Orientation 

III. C. 15 

Orientation 

IV. A. 7 

Two  2- level 

IV. A. 10 

One  3-level;  orientation 

Two  1-level 

IV. A. 12 

Orientation 

IV. B. 7 

Two  2-level 

IV.B.12 

One  3-level;  orientation 

Two  1-level 

IV.B.14 

Orientation 

IV. C. 7 

Two  2-level 

IV.C.13 

One  3-level;  orientation 

Two  1-level 

IV.C.15 

Orientation 

V.A.4 

Rewrite 

V.C.l 

Rewrite 

V.C.2 

Orientation 

♦One  or  more  2-level  changes/major  orientation  shift/rewrite 
♦♦One  or  more  1-level  changes/minor  orientation  shift/reword 
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Task  No.  *Disagreement 


TYPES  OF  CHANGES  (conf d) 

**Aqreement 


APPENDIX  E - Towboat  Operators 
TOW- 


I.B.4 

Two 

2-level 

I.B.8 

I.C.4 

Two 

2-level 

Orientation 

I.C.IO 

I.D.4 

Two 

2-level 

Orientation 

I.D.ll 
II. A. 7 

Two 

2-level 

Orientation 

II. A. 13 
II. B. 7 

Two 

2-level 

Orientation 

II.B.15 
II. C. 7 

Two 

2-level 

Orientation 

II.C.16 
III. A. 3 
III. A. 4 

Rewrite 

Orientation 

One  1-level;  orientation 

APPENDIX  F - 

Personnel  Handling  Ambient 

Pressure-Ambient  Temperature 

Hazardous  Chemical  Cargo  on  a Tankership 


I. A. 6 

One  1-level;  orientation 

II. A. 3 

Rewrite 

II. A. 4 

One  2-level  OR: 

One  1-level 

II. A. 9 

Rewording 

II. A. 12 

Orientation 

One  1 -level 

II. A. 13 

One  2- level 

II.A.16 

One  2-level 

One  1 -level 

II. A. 19 

One  2-level 

III. A. 2 

One  1-level;  orientation 

III. A. 6 

Orientation 

III.B.2 

One  1 -level 

III. C. 12 

One  1 -level 

III.D.3 

Orientation 

One  1 -level 

V.A.l 

Orientation 

V.A.2 

Orientation 

V.A.6 

Two  1 -level 

V.B.2 

"Things"  sub-category;  orientation 

V.B.7 

Two  1 -level 

V.C.l 

Orientation 

Three  1 -level 

V.C.2 

Orientation 

Three  1 -level 

V.C.3 

Orientation 

Three  1 -level 

V.D.l 

One  2-level 

Two  1 -level 

♦One  or  more  2-level  changes/major  orientation  shift/rewrite 
♦♦One  or  more  1-level  changes/minor  orientation  shifty/reword 
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TYPE  OF  CHANGES  (conf d) 


Task  No.  *Disagreeinent 


»*Agreement 


APPENDIX  G - Personnel  Handling  High  Pressure-Ambient  Temperature  Cargo 
on  an  Unmanned  Barge 


II. A. 2 

One  2-level; 

orientation 

II. A. 3 

One  2-level ; 

orientation 

Four  1 -level 

II.A.6 
II. B. 2 
III.C.8 
III.B.C 
III. A. 4 
III. D. 11 

Orientation 

Orientation 

Orientation 

One  1 -level 

One  1 -level 
One  1 -level 

One  1 -level 

III. D. 12 
III.E.4 
IV. A. 1 
IV. A. 4 
IV. A. 6 
IV.B.l 
IV. B. 6 

One  2-level 
Orientation 
Orientation 

One  1 -level 

One  1 -level 
Two  1 -level 
One  1 -level 
Two  1 -level 

IV.C.l 

Orientation 

Three  1 -level 

IV. C. 2 

Orientation 

Three  1 -level 

IV. C. 3 
IV.D.l 

Orientation 
One  2-level 

Three  1 -level 

♦One  or  more  2-level  changes/major  orientation  shift/rewrite 
♦♦One  or  more  1 -level  changes/minor  orientation  shift/reword 
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